Hi,

1. As the other prior two PRs in common-utils and ccr are not yet merged, the checks on CI would not go through. Also, in build.gradle, locally placed artifacts for these projects are being used. I shall remove those changes once the artifacts are merged and available in repo.
2. I've also added a UT (AttemptUnfollowStepsTest.kt) - but that is WIP as its not working correctly, as I've discussed with @bowenlan-amzn. I've tried two approaches to mock the response value, by using mockStatic (to mock ReplicationInterface) and mock spy (to mock AttemptUnfollowStep.performStopAction() alone) but havent been successful.
   Would like to hear if there any suggestions in this area.

Looking forward for the review, Thanks!

Hi Reviewers,
As suggested by @bowenlan-amzn, I'm sharing some details on how one can bring up the environment with ccr and ism for this use-case. I've done this for a linux environment.

Environment setup details

1. Setup Opensearch

$ git clone https://github.com/opensearch-project/OpenSearch.git 
$ cd Opensearch
### Ensure JAVA_HOME, PATH are set. 
$ ./gradlew assemble
$ export OPENSEARCH_HOME=<opensearch-home-dir>   ## where you wish to setup Opensearch
$ export OPENSEARCH_BUILD=distribution/archives/linux-tar/build/install/opensearch-$(./gradlew properties -q | grep -E '^version:' | awk '{print $2}')
$ cp -Rf $OPENSEARCH_BUILD/* $OPENSEARCH_HOME

(Above steps can be avoided if you already have opensearch setup)
Installation of plugins:

 $ cd $OPENSEARCH_HOME
 $ ./bin/opensearch-plugin install <location-of-opensearch-job-scheduler-3.0.0.0-SNAPSHOT.zip>
 $ ./bin/opensearch-plugin install <location-of-opensearch-cross-cluster-replication-3.0.0.0-SNAPSHOT.zip>
 $ ./bin/opensearch-plugin install <location-of-opensearch-index-management-3.0.0.0-SNAPSHOT.zip>

For ex. bin/opensearch-plugin install file:///home/username/ccr/opensearch-cross-cluster-replication-3.0.0.0-SNAPSHOT.zip

2. Setup Configs
We need to bring up one cluster as the leader and other as the follower (that replicates the leader indices).
This small snippet is a function to generate opensearch.yml config files for the two clusters. Run this from $OPENSEARCH_HOME location.

configure_opensearch() {
	 role=$1
	 mkdir -p ccrtest/$role/data/  mkdir -p ccrtest/$role/logs
	 cp config/opensearch.yml config/opensearch-$role.yml
	 echo "
	 plugins.security.disabled: true
	 cluster.name: ${role}Cluster
	 path.data: $OPENSEARCH_HOME/ccrtest/${role}/data
	 path.logs: $OPENSEARCH_HOME/ccrtest/${role}/logs
	
	 " >> config/opensearch-${role}.yml
}
	
// Invoke the function twice. It generates files opensearch-leader.yml and  opensearch-follower.yml at $OPENSEARCH_HOME/config.
configure_opensearch "leader"
configure_opensearch "follower"

(This is using the default opensearch.yml file and just adding minimal required configs for CCR)

3. Run Opensearch clusters
   Leader cluster as 127.0.0.1:9200, follower as 127.0.0.1:9201 using the generated config files respectively in step 2.

4. Start Replication using CCR REST APIs
   Create an index in leader cluster, and setup replication in follower cluster.

5. Start ISM
   Create an ism policy to run the action "unfollow" on the desired indices.

For steps 3-5, I've added the steps in a shell script for convenience - test-unfollow-script.txt. (Changed the extension to .txt as .sh files cannot be uploaded here)

I hope that helps : )

Hi all
Status update:
Changes for this are ready to be pushed. However, we have a blocking issue recently. Details below:

• The common-utils project has moved to using opensearch-3.0.0-alpha1-SNAPSHOT. One breaking change here is that classes like AcknowledgedRequest, AcknowledgedResponse etc are moved from org.opensearch.action.support.master to org.opensearch.action.support.clustermanager.
• ccr and ism projects are still using opensearch-3.0.0-SNAPSHOT and classes from old org.opensearch.action.support.master package. This inconsistency results in incompatibility issues while using these classes.
• As discussed over slack with @bowenlan-amzn and @ankitkala, it is suggested to raise PRs on 2.x branches first, until the development issue with the main branches is resolved.
• For this change, I've raised a backport PR on 2.x branch ([2.x] Adding new stop-replication action in ism #1370)

Have reviewed all the PRs, and setup my local run on 2.19.1 and able to succeed.
Great job! Thank you for seeing it through to the end!

Have reviewed all the PRs, and setup my local run on 2.19.1 and able to succeed. Great job! Thank you for seeing it through to the end!

Thank You for the review and valuable feedback throughout
For the main branch, all review comments received so far have been addressed and PRs are ready for review - this and common-utils and ccr too.

Pls note:
1] CI Checks in ccr and ism would fail until common-utils is merged.
2] For CCR plugin publishing though, still need to see if this change is sufficient.